Support Vector Machines versus Boosting
نویسندگان
چکیده
Support Vector Machines (SVMs) and Adaptive Boosting (AdaBoost) are two successful classification methods. They are essentially the same as they both try to maximize the minimal margin on a training set. In this work, we present an even platform to compare these two learning algorithms in terms of their test error, margin distribution and generalization power. Two basic models of polynomials and decision stumps are used to evaluate eight real-world binary datasets. We concluded that the generalization power of AdaBoost with linear SVMs as base learners and that of SVMs with decision stumps as kernels outperform other scenarios. Although the training error of AdaBoost approaches to zero with the increase number of weak learners, its test error starts to rise at a certain step. For both SVMs and AdaBoost, the cumulative margin distribution is indicative of the test error.
منابع مشابه
Boosting Support Vector Machines
This paper presents a classification algorithm based on Support Vector Machines classifiers combined with Boosting techniques. This classifier presents a better performance in training time, a similar generalization and a similar model complexity but the model representation is more compact.
متن کاملA Hybrid Random Forest based Support Vector Machine Classification supplemented by boosting
This paper presents an approach to classify remote sensed data using a hybrid classifier. Random forest, Support Vector machines and boosting methods are used to build the said hybrid classifier. The central idea is to subdivide the input data set into smaller subsets and classify individual subsets. The individual subset classification is done using support vector machines classifier. Boosting...
متن کاملLeveraged Vector Machines
We describe an iterative algorithm for building vector machines used in classification tasks. The algorithm builds on ideas from support vector machines, boosting, and generalized additive models. The algorithm can be used with various continuously differential functions that bound the discrete (0-1) classification loss and is very simple to implement. We test the proposed algorithm with two di...
متن کاملA Comparative Study of Extreme Learning Machines and Support Vector Machines in Prediction of Sediment Transport in Open Channels
The limiting velocity in open channels to prevent long-term sedimentation is predicted in this paper using a powerful soft computing technique known as Extreme Learning Machines (ELM). The ELM is a single Layer Feed-forward Neural Network (SLFNN) with a high level of training speed. The dimensionless parameter of limiting velocity which is known as the densimetric Froude number (Fr) is predicte...
متن کاملBoosting as a Regularized Path to a Maximum Margin Classifier
In this paper we study boosting methods from a new perspective. We build on recent work by Efron et al. to show that boosting approximately (and in some cases exactly) minimizes its loss criterion with an l1 constraint on the coefficient vector. This helps understand the success of boosting with early stopping as regularized fitting of the loss criterion. For the two most commonly used criteria...
متن کاملA Short Introduction to Boosting
Boosting is a general method for improving the accuracy of any given learning algorithm. This short overview paper introduces the boosting algorithm AdaBoost, and explains the underlying theory of boosting, including an explanation of why boosting often does not suffer from overfitting as well as boosting’s relationship to support-vector machines. Some examples of recent applications of boostin...
متن کامل